Soft thresholding soft(x,T):={x+Tx≤−T0|x|≤Tx−Tx≥T \mathrm{soft}(x,T) := \begin{cases} x + T & x \leq -T \\ 0 & |x| \leq T \\ x - T & x \geq T \end{cases} See also: https://en.wikipedia.org/wiki/LogSumExp https://en.wikipedia.org/wiki/Rectifier_(neural_networks)#Softplus